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METHOD AND APPARATUS FOR COLOR 
COMPARISON IN A DIGITAL CAMERA 

FIELD OF THE INVENTION 
5 The present invention relates generally to digital cameras and more 

specifically to a method and associated apparatus for comparing colors in a digital 
camera. 

BACKGROUND OF THE INVENTION 
10 People sometimes encounter colors they like on everyday objects or in 

nature. For example, someone may see a parked car, house, curtain, garment, or 
other object containing a color that he or she finds especially attractive. The 
observer may desire to remember the color and match it later at, for example, a 
paint or fabric store. Since taking a sample of the color from the object is often not 
15 feasible, one solution is to take a picture of the object of interest. However, taking 
a picture using a traditional silver-emulsion-film camera is an imprecise method to 
match colors due to such factors as lens filters, exposure errors, and film processing 
variability and imperfections. Differences in illuminant and illumination level 
between the photograph and the color specimens with which it is ultimately 
20 compared further complicates the problem. For example, colors typically appear 
different under fluorescent lighting than under incandescent lighting. A digital 
camera provides greater convenience and immediate feedback than traditional 
photography, but the difficulties with illuminant and illumination level apply just as 
much to digital photography as to the conventional type. 
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It is thus apparent that there is a need in the art for an improved method or 
apparatus for remembering and comparing colors. 

SUMMARY OF THE INVENTION 
5 A method is provided for storing a reference color in a digital camera and 

for comparing the color of a candidate specimen with the reference color. A digital 
camera is also provided for carrying out the methods. 

Other aspects and advantages of the present invention will become apparent 
from the following detailed description, taken in conjunction with the accompanying 
10 drawings, illustrating by way of example the principles of the invention. 



BRIEF DESCRIPTION OF THE DRAWINGS 
FIG. 1 is a functional block diagram of a digital camera in accordance with the 
present invention. 

15 FIG. 2 is a flowchart of the operation of the digital camera shown in FIG. 1 in 

accordance with one aspect of the present invention. 

FIG. 3A is an illustration of one method for specifying a color-analysis sub- 
image in the digital camera shown in FIG. 1. 

FIG. 3 B is an illustration of another method for specifying a color-analysis 
20 sub-image in the digital camera shown in FIG. 1 . 

FIG. 4 is a flowchart of the operation of the digital camera shown in FIG. 1 in 
accordance with another aspect of the invention. 

FIG. 5A is an illustration of one method for reporting the difference between 
the color of a candidate specimen and a reference color in accordance with another 
25 embodiment of the present invention. 
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FIG. 5B is an illustration of a different method for reporting the difference 
between the color of a candidate specimen and a reference color in accordance with 
yet another embodiment of the present invention. 

5 DETAILED DESCRIPTION OF THE INVENTION 

FIG. 1 is a functional block diagram of a digital camera 100 in accordance 
with one exemplary embodiment of the present invention. In FIG. 1, user interface 
105 comprises input device 110, display 115, and optional audible tone generator 
120. Input device 110 sends commands to controller 125 over data bus 130 to 

10 specify the modes of operation of digital camera 100. User interface 105 provides 
feedback to users via display 115 and optional audible tone generator 120. Imaging 
device 135 converts an optical image of a subject received from optical system 140 
to a digital image. The digital image may be stored temporarily in optional random 
access memory (RAM) 145. Illuminant normalization module 150 compensates for 

15 different illuminants from one scene to another to make accurate color comparison 
possible. Non-volatile memory 155 provides long-term storage of color descriptions 
and digital images. Optional flash unit 160 may be used to illuminate subjects with 
a known illuminant and to compensate for low-light conditions. 

One example of how the illuminant of a digital image may be estimated and 

20 corrected may be found in U.S. Patent 6,038, 339, "White Point Determination 
Using Correlation Matrix Memory," assigned to Hewlett-Packard Company, the 
disclosure of which is herein incorporated by reference. In U.S. Patent 6,038,339, 
a correlation matrix memory is built to correlate the data from any digital image 
with reference images under a range of illuminants. The digital image data from 
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digital camera 100 is converted to chromaticity coordinates, and a vector is created 
corresponding to the values existing in the scene. This vector is multiplied by each 
column in the correlation matrix, yielding a new matrix. Each column is then 
summed, and the resulting values form a vector, the components of which represent 

5 the likelihood of each reference source being the illuminant of the scene. The 

vector values can be subsequently density plotted, where each value is plotted at the 
chromaticity of the illumination for that particular column. From this plot, normal 
statistical methods may be used to estimate the likely illuminant of the scene. Once 
the likely illuminant has been estimated, the spectral characteristics of the illuminant 

10 may be compensated to normalize the illuminant of the digital image to approximate 
a standard illuminant, to be discussed in more detail in a later portion of this 
detailed description. 

A variety of different color representations may be used with the present 
invention. One method for describing colors that is well known in die art is the use 

15 of red, blue, and green (RGB) color components or coordinates, sometimes referred 
to as tristimulus values. In the RGB representation of a digital image, each pixel is 
represented by RGB components, each of which is represented by, typically, 8-12 
bits in digital camera applications. As those skilled in the art will recognize, the 
RGB color space is not desirable for colorimetric calculations such as color 

20 differences because it is non-linear with respect to human visual perception. To 
compare colors accurately, a uniform color space is desirable in which equal 
distances in any direction in the three-dimensional space correspond to color 
differences of approximately equal perceptual magnitude. One well-known color 
space that is approximately uniform for small changes in color is the standard 
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Commission Internationale de l'Eclairage (CIE) L* u* v* color space comprising a 
luminance and two chrominance coordinates, respectively. However, other 
approximately uniform color spaces are possible such as L* a* b* 7 which is popular 
in the textiles industry. The application of the present invention is not limited to any 

5 particular color representation. Also, throughout this detailed description, "saving" 
or "storing" a color will denote storing in non-volatile memory 155 a specification 
of the color (i.e., color coordinates), a digital image from which the color was 
determined, or both. 

In a typical implementation, imaging device 135 comprises a charge-coupled 

10 device (CCD), an analog-to-digital converter (A/D), a gain control, and a digital 
signal processor (DSP), as is well known in the art (not shown in FIG. 1). Input 
device 110 typically comprises one or more buttons for selecting modes and options 
in digital camera 100. Audible tone generator 120 is an optional component of 
digital camera 100 and has been included in FIG. 1 for the purpose of illustration 

15 only. In some implementations, a flashing or persistent light, a vibrator, or other 
method of providing feedback to users may be preferable to an audible tone. 
Illuminant normalization module 150 may be implemented in hardware, software, 
firmware, or a combination thereof. Non-volatile memory 155 is typically flash 
memory but may, in some implementations, be of the removable type, such as a 

20 memory stick or magnetic disk. 

FIG. 2 is a flowchart of the operation of digital camera 100 in accordance 
with one aspect of the present invention, a method for storing a reference color in a 
digital camera. At 205, controller 125 determines whether or not color-storage mode 
has been activated in digital camera 100. If so, control proceeds to 210. In 
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attempting to isolate a particular color contained in scene, it is sometimes helpful to 
restrict color analysis to a relatively small sub-region of the scene. Thus, at 210 an 
optional color-analysis sub-image may be specified on display 115 using input device 
110. More detail regarding the specification of a color-analysis sub-image will be 
5 provided later in this detailed description. Once a specimen containing a color of 
interest has been identified, a digital image of the specimen is captured at 215. The 
digital image may be stored temporarily in RAM 145. The use of optional flash unit 
160 in capturing the digital image at 215 provides a known illuminant, making 
subsequent color comparison easier and more precise. If no flash is used, illuminant 
10 shortcomings may be ignored, and the present invention may still be used to store and 
compare colors. An alternative, if no flash is used, is to estimate and normalize the 
illuminant of the digital image, shown at 220 in connection with this particular 
embodiment of the invention. Illuminant normalization module 150 estimates the 
illuminant present in the digital image and produces a normalized version of the 
15 digital image based on a standard illuminant. For example, the illuminant may be 
corrected to one of several CIE standard illuminants such as Illuminant A (gas-filled 
tungsten lamp at 2848°K) or Illuminant C (average daylight), which are well known 
in the art. The normalized digital image produced at 220 may also be stored 
temporarily in RAM 145. Even though the specimen may appear to contain only one 
20 color, not every pixel in the digital image will necessarily have identical color 
components (e.g., RGB or 

£* v*). Therefore, the normalized digital image is analyzed at 225 to determine an 
aggregate color within the optional color-analysis sub-image specified at 210. Many 
possible statistical measures may be used in determining an aggregate color. 
25 Examples include the mean, the median, or the mode of the color components within 
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the color-analysis sub-image. Once an aggregate color has been determined at 225, it 
is saved as a reference color in non-volatile memory 155 at 230. If additional 
reference colors are to be stored at 235, control returns to 210. Otherwise color- 
storage mode is exited at 240. Optionally, a user may name each stored reference 
5 color using input device 100, or digital camera 100 may automatically generate 
unique names for the stored reference colors such as "Color 1," "Color 2," and so 
forth. Using input device 1 10, a list of stored reference colors may be recalled, and a 
specific stored reference color may be selected from the list as the active reference 
color for subsequent comparison and matching. Color comparison will be explained 
10 in a later portion of this detailed description. 

Instead of obtaining the reference color as described in connection with FIG. 
2, the reference color may instead be obtained in the form of color coordinates from 
an external source, without the need to capture a digital image of a specimen. For 
example, color coordinates specifying a particular color may be input directly via 
15 input device 1 10, or color coordinates may be downloaded to digital camera 1 10 from 
a different external source such as a personal computer, which may in turn be 
connected to the Internet. In this implementation, it is possible to obtain color 
descriptions from the Internet, download them to digital camera 100, and search for 
matching colors in candidate specimens. Communication interfaces between digital 
20 cameras and, for example, desktop or laptop computers capable of supporting this 
implementation are well known in the art. 

The optional color-analysis sub-image that may be specified at 210 is that 
portion of the normalized digital image that is analyzed in determining the aggregate 
color, which ultimately becomes the saved reference color. All pixels outside the 
25 color-analysis sub-image are ignored for the purpose of determining the aggregate 
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color. FIG. 3A illustrates one approach to specifying a color-analysis sub-image. 
Display 1 15 on digital camera 100 includes bounding box 305, the interior of which 
comprises the color-analysis sub-image. Bounding box 305 may appear automatically 
on display 1 15 whenever color-comparison mode is activated. Only that portion of 

5 specimen 310 lying within bounding box 305 is included in the determination of the 
aggregate color at 225. Controller 125 interacts with input device 1 10 and display 
1 15 containing bounding box 305 to determine which portion of the normalized 
digital image to extract for determining the aggregate color. However, those skilled 
in the art will recognize that greater accuracy in the estimation of the illuminant at 

10 220, when it is necessary, may be obtained by including all pixels in the normalized 
digital image, not only those within the color-analysis sub-image. 

In a different embodiment of the color-storage aspect of the present invention, 
bounding box 305 is replaced by a more precise fiducial such as a moveable crosshair. 
In this embodiment, the color-analysis sub-image is restricted to just a few pixels 

15 lying beneath the crosshair, providing for very precise pinpointing of a specific color 
within a scene. Further, since the color-analysis sub-image comprises only a few 
pixels, the computational burden on controller 125 in determining the aggregate color 
is reduced. FIG. 3B illustrates one possible implementation of this embodiment. 
Crosshair 315 replaces bounding box 305 in defining the portion of specimen 310 

20 comprising the color-analysis sub-image. 

FIG, 4 is a flowchart of the operation of digital camera 100 in accordance 
with another aspect of the invention, that dealing with color comparison. Controller 
125 determines at 405 whether a color-comparison mode has been activated in digital 
camera 100. If so, an optional color-analysis sub-image is specified at 210 as 

25 explained in connection with FIG. 2. Once a candidate specimen has been identified 
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for comparison with the currently selected reference color, a digital image of the 
candidate specimen is captured at 410. Image capture at 410 may be automatic or 
manually initiated by the user. As explained in connection with FIG. 2, flash is one 
effective method for providing a known illuminant in the image captured at 410. It is 

5 desirable, though not essential, that flash be used in producing the digital images from 
which colors to be compared are derived. However, the present invention may be 
used with or without flash. At 220, the illuminant of the digital image is optionally 
normalized, and an aggregate color is determined at 225, as explained in connection 
with FIG. 2. Controller 125 computes the difference between the aggregate color and 

10 the currently selected reference color at 415. Optionally, the difference is also 

reported at 415. More will be said about reporting the difference between the two 
colors later in this detailed description. Although many different colorimetric 
methods exist for comparing colors, one suitable definition of the difference between 
two colors is the following: 

15 d = A /(z*2"L*l) 2 + (tt*2-K*J + (v*2-V*J > Equation 1 

where (i*i,w*i,v*i) are the L*, m*, v* components of the stored reference color, 
(X*2,w*2,v* 2 ) are the L* 9 u* 9 v* components of the aggregate color associated with the 
candidate specimen, and d is the difference between the two colors. Those skilled in 
the art will recognize Equation 1 as the CIE 1976 (Z* w* v*) color difference or 

20 CIELUV. The difference d is compared with a predetermined tolerance T at 420. The 
tolerance J may be fixed, or it may be adjusted to fit different applications. The 
tolerance T may be specified in absolute terms for direct comparison with d, as per 
cent error between the reference color and the aggregate color of the candidate 
specimen, or as a multiple of a standard minimum perceptible color difference 

25 (MPCD). The MPCD is a well-known concept in color science corresponding to the 
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"just perceptible color difference" between two colors as they are compared under 
standard conditions. Those skilled in the art will recognize that per cent error can 
easily be converted to a tolerance T in units compatible with d in Equation 1 . In a 
typical implementation, the user may select from among a set of possible choices such 
5 as "close," "very close," or "best possible" match, each of which may correspond to a 
specific per cent error or multiple of the MPCD. For example, a "best possible" 
match may correspond to one MPCD, and a "close" match may correspond to five 
MPCDs. Ifd is less than Tat 420, a match is signaled at 425. Signaling the match 
may comprise visual feedback on display 1 15, one aspect of which will be explained 
10 more fully in a later portion of this description, audible feedback such as a beep or 
tone, or other forms of feedback such as vibration, a flashing light, a persistent light, 
or a text message on display 115 such as "match found." At 430, additional candidate 
specimens may be compared with the reference color by return to 210, or color- 
comparison mode may be exited at 435. 
15 A variety of techniques may be used to report the difference d at 415. One 

straightforward approach is simply to display a number on display 115 indicating the 
closeness of the match. The number may be d, multiples of the MPCD, per cent error, 
or some other appropriate figure of merit. Alternatively, a text string such as "not 
close," "close," or "very close " may be shown on display 115. Another effective 
20 approach is shown in FIG. 5 A, where display 1 15 is divided into two regions, 505 and 
510. Region 505 is a smaller version of the normal display, which displays the 
current scene received from optical system 140 or, in some situations, a captured 
digital image. Bounding box 305 defines the color-analysis sub-image associated 
with the current color difference indication provided in region 510. Alternatively, a 
25 crosshair may be used to define a more precise color-analysis sub-image as explained 
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in connection with FIG. 3B. Region 510 contains a color wheel 515 such as those 
well known in the art and commonly used in computer image editing and graphics 
applications. Tolerance Tis indicated on color wheel 515 as a circular boundary 
concentric with stored reference color 525. Aggregate color 530 is also shown in its 

5 applicable position on color wheel 515. In the particular example illustrated in FIG. 
5 A, a match has been found. That is, the aggregate color is within T of the reference 
color. One alternative to a color wheel is a color map, which is typically square or 
rectangular in shape. A color map is essentially a two-dimensional matrix containing 
rows and columns of color samples. In this case, tolerance Tmay be displayed as a 

10 square boundary concentric with the reference color on the color map. 

Alternatively, a linear approach may be used to provide a larger region 505 for 
displaying scenes, as shown in FIG. 5B. In FIG. 5B, region 535 contains a linear 
gauge to report the difference d. Line 540 represents a continuum of colors, from 
those unlike the stored reference color at one end to the precise stored reference color 

15 itself at the opposite end. The bottom end of line 540 represents the set of all colors 
that differ from the stored reference color by greater than a predetermined amount. 
For example, the bottom end of line 540, in one implementation, may represent all 
colors that differ from the stored reference color by more than 50 MPCDs. The 
opposite end of line 540 represents the stored reference color, which is indicated as 

20 dot 545 in FIG. 5B. Open circle 550 represents the aggregate color associated with 
the candidate specimen, and tolerance Tis represented as perpendicular line 555. 
Thus, line 540 may be thought of as comprising two portions divided by 
perpendicular line 555. The portion of line 540 above perpendicular line 555 
represents the set of all colors that differ from the reference color by less than 7, and 

25 the remainder of the line represents the set of all colors that differ from the reference 
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color by an amount greater than or equal to T. Those skilled in the art will recognize 
that there are many alternative ways in which to arrange a linear gauge such as that 
shown in region 535 of FIG. 5B. Variations include, but are not limited to, horizontal 
rather than vertical orientation, different markings to indicate the colors being 
5 compared, and a different mark to indicate tolerance T. Those skilled in the art will 
also recognize that, if the difference between the aggregate color associated with a 
candidate specimen and the stored reference color exceeds the range of the linear 
gauge, line 540 may be rescaled to include whatever difference is computed at 415. 
Conversely, as the aggregate color associated with each of a series of candidate 
10 specimens more closely approaches the reference color, line 540 may be rescaled to 
encompass a smaller range of color differences (e.g., 20 MPCDs), thereby improving 
the resolution of the reported difference. 

Another embodiment of the color comparison aspect of the present invention 
provides for multiple color matches within a single color-analysis sub-image. In this 
15 embodiment, the color-analysis sub-image may be larger than in the embodiment 
discussed previously, possibly comprising the entire normalized digital image. 
Controller 125 analyzes the color-analysis sub-image associated with the candidate 
specimen to identify regions of similar color. For each such region, controller 125 
determines an aggregate color. Those regions of the normalized digital image for 
20 which the associated aggregate color differs from the stored reference color by less 
than tolerance Tare shown at normal intensity on display 115. Those regions for 
which the associated aggregate color differs by Tor more are shown at reduced 
intensity on display 115. For example, the regions for which no color match occurs 
may be shown at half intensity. 
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In some situations, comparing colors is complicated by overexposure 
(illumination level too high) or underexposure (illumination level too low). 
Illumination level compensation may be added as an additional optional feature in any 
embodiment of the present invention. One implementation is for controller 125 to 

5 convert RGB components to chromaticity coordinates and to compare only the 

chrominance portion. In most cases, however, unless a digital image is grossly under- 
or overexposed, illuminant normalization is sufficient to accomplish the purposes of 
the present invention. 

The foregoing description of the present invention has been presented for the 

10 purposes of illustration and description. It is not intended to be exhaustive or to limit 
the invention to the precise form disclosed, and other modifications and variations 
may be possible in light of the above teachings. The embodiments were chosen and 
described in order to best explain the principles of the invention and its practical 
application to thereby enable others skilled in the art to best utilize the invention in 

15 various embodiments and various modifications as are suited to the particular use 
contemplated. It is intended that the appended claims be construed to include other 
alternative embodiments of the invention except insofar as limited by the prior art. 
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